Systems and methods for generating fused medical images from multi-parametric, magnetic resonance image data

ABSTRACT

This invention provides a system and method for fusing and synthesizing a plurality of medical images defined by a plurality of imaging parameters allowing visual enhancements of each image data set to be combined. The system provides an image fusion process/processor that fuses a plurality of magnetic resonance imaging datasets. A first image dataset of the datasets is defined by apparent diffusion coefficient (ADC) values. A second image dataset of the MRI datasets is defined by at least one parameter other than the ADC values. The image fusion processor generates a fused response image that visually displays a combination of image features generated by the ADC values combined with image features generated by the at least one parameter other than the ADC values. The fused response image can illustratively include at least one of color-enhanced regions of interest and intensity-enhanced regions of interest.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/927,039, filed Jun. 25, 2013, entitled SYSTEMS AND METHODS FORGENERATING FUSED MEDICAL IMAGES FROM MULTI-PARAMETRIC, MAGNETICRESONANCE IMAGE DATA, which is now abandoned, which is a continuation ofco-pending U.S. patent application Ser. No. 12/797,231, filed Jun. 9,2010, now U.S. Pat. No. 8,472,684, issued Jun. 25, 2013, entitledSYSTEMS AND METHODS FOR GENERATING FUSED MEDICAL IMAGES FROMMULTI-PARAMETRIC, MAGNETIC RESONANCE IMAGE DATA, the entire disclosuresof each of which are herein incorporated by reference.

FIELD OF THE INVENTION

This application relates generally to the processing of discrete groupsof medical image data. More particularly, this application relates toautomated techniques for fusing and/or synthesizing medical image dataacquired by imaging tissue with multiple scanning parameters, sequences,or protocols.

BACKGROUND OF THE INVENTION

Early detection of disease and malignant tissue can lead to a betterprognosis. The development of non-invasive methods for detection andcharacterization of tumors has an extreme importance in currentmedicine. Magnetic resonance imaging (MRI) is a noninvasive medical testthat can help physicians diagnose and treat medical conditions. MRimaging uses a powerful magnetic field, radio frequency pulses and acomputer to produce detailed pictures of organs, soft tissues, bone andvirtually all other internal body structures. The images can be examinedon a computer monitor, printed or copied to compact disc.

By way of useful background information, according to Breast MRI:fundamentals and technical aspects, Hendrick, New York, N.Y.: Springer(2008), one effective breast MRI protocol for the detection anddiagnosis of breast cancer has several essential elements: anon-fat-saturated T1-weighted pulse sequence, a fat-saturatedT₂-weighted pulse sequence, and a set of contrast-enhanced pulsesequences at different phases to obtain identically acquiredpre-contrast and multiple post-contrast views.

From a workflow standpoint, a clinician typically views MR imagedatasets by loading and outputting them to a computer, for display on acomputer monitor or series of computer monitors. In this manner, thephysician can visually study and manipulate the various images or“softcopies” of the images. The clinician often pre-defines a hangingprotocol, which refers to the clinician's preferred arrangement ofimages for optimal softcopy viewing. Different clinicians might preferto review images in different manners, depending on their experience andpersonal preferences. Synthesizing the information across images can bea laborious process for the clinician, especially due to the largenumber of thin slice images provided in each MR image dataset and theamount of spatial information in one dataset that might need to becorrelated with information from one or more other datasets.

Some researchers have developed fully automated multi-modal MM fusiontechniques. By way of useful background information, one such example isdescribed in INTEGRATING STRUCTURAL AND FUNCTIONAL IMAGING FOR COMPUTERASSISTED DETECTION OF PROSTATE CANCER ON MULTI-PROTOCOL IN VIVO 3 TESLAMRI, by Viswanath et al, Proc. SPIE 7260, 726031 (2009).Disadvantageously, prior art fusion methods such as those described inViswanath, provide preset solutions (e.g., fusion parameters) that areapplied to all datasets regardless of the clinician's individual needsand desires and fail to integrate certain parameters of use in formingfusion images that aid clinician's in differentiating tissue types.These approaches, thus, lack both intuitiveness for a clinician andflexibility in adapting to a clinician's specific protocol.

SUMMARY OF THE INVENTION

This invention overcomes disadvantages of the prior art by providing asystem and method for fusing and synthesizing a plurality of medicalimages defined by a plurality of imaging parameters that allow thevisual enhancements of each image data set to be selectively combinedwith those of other image datasets. In this manner, a user-definedparameter set can be generated in the final response image dataset. Thisfinal response image dataset displays visual data represents a formparticularly useful to the clinician. In an illustrative embodiment, thesystem for fusing and synthesizing the plurality of medical imagesprovides an image fusion process/processor that fuses a plurality ofmagnetic resonance imaging (MRI) datasets. A first image dataset of theMRI datasets is defined by apparent diffusion coefficient (ADC) values.A second image dataset of the MRI datasets is defined by at least oneparameter other than the ADC values. The image fusion processorgenerates a fused response image that visually displays a combination ofimage features generated by the ADC values combined with image featuresgenerated by the at least one parameter other than the ADC values. Thefused response image can illustratively include at least one ofcolor-enhanced regions of interest and intensity-enhanced regions ofinterest.

In an illustrative embodiment, the at least one parameter can be basedupon at least one of a T₂-weighted medical image and a dynamic contrastenhanced MRI (DCE-MRI) medical image. A registration process/processorcan be provided. It aligns each of the first image data set and thesecond image dataset into a registered multi-modal image dataset. Thisregistration process/processor can also illustratively include anon-rigid registration process/processor and an atlas/template processorthat operates upon the image data based upon atlas coordinate datarelated to imaged tissue. In an embodiment, a segmentationprocess/processor is provided to apply organ/tissue atlas coordinatedata to the registered multi-modal image dataset to generate segmentedorgan/tissue image data with respect to regions of interest in themulti-modal image dataset. Additionally, an intensity homogeneitycorrection process/processor can be provided to generate a homogeneousorgan/tissue image dataset by smoothing and filtering image intensitieswith respect to the organ/tissue image data.

In an embodiment, the fusion process/processor receives inputs ofuser-defined parameters to vary image data displayed in the fusedresponse image in accordance with the user's desired criteria. Thesecriteria can be input in advance of any processing of the image data, orduring image processing operations. The user can visually observe howthe variation of parameters changes the output results, and adjust theuser-defined parameters accordingly during runtime of the system. Thefusion process/processor can further include (a) a scale normalizingprocess/processor that receives map data from a multi-modal parametersource and generates scale-normalized parameter values and (b) aresponse process/processor that generates response values that definethe fused response image.

In another embodiment, a system and method for fusing and synthesizing aplurality of medical images defined by a plurality of imaging parametersincludes an image fusion process/processor that fuses a plurality ofmagnetic resonance imaging (MRI) datasets, in which a first imagedataset of the MRI datasets is defined by at least a first parameter anda second image dataset of the MRI datasets is defined by at least asecond parameter. The image fusion process/processor generates a fusedresponse image that visually displays a combination of image featuresfrom the first image dataset and image features from the second imagedataset based upon user defined parameters input to the image fusionprocess/processor through a user interface prior to operation of theimage fusion process/processor to generate the fused response image.

In yet another embodiment, a system and method for fusing andsynthesizing a plurality of medical images defined by a plurality ofimaging parameters includes an image fusion process/processor that fusesa plurality of magnetic resonance imaging (MRI) datasets, in which afirst image dataset of the MRI datasets is defined by morphology valuesand a second image dataset of the MRI datasets is defined by at leastone parameter other than the morphology values. The image fusionprocess/processor generates a fused response image that visuallydisplays a combination of image features from the first image datasetand image features from the second image dataset.

In yet another embodiment, a system and method for fusing andsynthesizing a plurality of medical images defined by a plurality ofimaging parameters includes an image fusion process/processor that fusesa plurality of magnetic resonance imaging (MRI) datasets. The imagefusion process/processor generates a fused response image that visuallydisplays a combination of image features from the first image datasetand image features from the second image dataset. Illustratively asegmentation processor, in communication with the image fusionprocessor, generates corresponding segments of each of the first imagedataset and the second image data set based upon predeterminedsegmenting data so that each of the segments, so that the correspondingsegments are each discretely fused by the image fusion processor.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention description below refers to the accompanying drawings, ofwhich:

FIG. 1 is a block diagram of an illustrative embodiment of an imageprocessing system;

FIG. 2 is a depiction of illustrative anatomical regions or zones of aprostate that can be segmented and processed using the image processingsystem of FIG. 1;

FIG. 3 is a block diagram of an illustrative embodiment of an imagefusion process/processor for use with the image processing system ofFIG. 1;

FIG. 4 is a depiction of an exemplary T₂-weighted intensity map that canbe acquired and processed using the image fusion process/processor ofFIG. 3;

FIG. 5 is a depiction of an exemplary apparent diffusion coefficient(ADC) map that can be acquired and processed using the image fusionprocess/processor of FIG. 3;

FIG. 6 is a depiction of an exemplary permeability map that can beacquired and processed using the image fusion process/processor of FIG.3;

FIG. 7 is a depiction of an exemplary extracellular volume map that canbe acquired and processed using the image fusion process/processor ofFIG. 3;

FIG. 8 is a depiction of an exemplary efflux rate constant map that canbe acquired and processed using the image fusion process/processor ofFIG. 3;

FIG. 9 is a flow diagram showing an illustrative runtime image fusionprocedure for use with the image fusion process/processor of FIG. 3; and

FIG. 10 is a depiction of an exemplary multi-parametric fusion imagethat can be generated and output using the image processing system ofFIG. 1.

DETAILED DESCRIPTION

In the present disclosure, the terms “pixels” and “voxels” can be usedinterchangeably to refer to an element in an image. Image data isgenerally represented in units of picture elements (pixels). A pixelgenerally refers to the information stored for a single grid in an imageor a basic unit of the composition of an image, usually in atwo-dimensional space, for example, x-y coordinate system. Pixels canbecome volumetric pixels or “voxels” in three-dimensional space (x, y, zcoordinates) by the addition of at least a third dimension, oftenspecified as a z-coordinate. A voxel thus refers to a unit of volumecorresponding to the basic element in an image that corresponds to theunit of volume of the tissue being scanned. It should be appreciatedthat this disclosure can utilize pixels, voxels and any other unitrepresentations of an image to achieve the desired objectives presentedherein. Both pixels and voxels each contain a discrete intensity and/orcolor, which is typically defined as one or more digital values within agiven range (for example, a grayscale intensity between 0 and 255, ordiscrete RGB values each between 0 and 255).

Also in the present disclosure, the terms “image”, “dataset” and/or“image dataset” can be used interchangeably to refer not just to asingle image, but to an n-dimensional plurality of images. These imagescan take the form of a volume of images, a plurality of volumes, or evena plurality of datasets. By way of example, in dynamic,contrast-enhanced magnetic resonance imaging (DCE-MRI), a plurality ofslice images is typically acquired before, during, and after contrastagent infusion, resulting in the acquisition of a time sequence of imagevolumes. In this example, the terms “image”, “dataset” and/or “imagedataset” can be used to refer to a plurality of slice images of thetissue at a given time point, a plurality of slices images of the tissueacross different time points, or a plurality of image volumes of thetissue across different time points.

With reference to FIG. 1, there is shown a block diagram of componentsof an image processing system 100 according to an illustrativeembodiment. Processor blocks shown within the system illustratedifferent image processing functions that can be performed on medicalimage data. Such functions can be realized by suitable combinations ofhardware and software components (“software” being defined herein as acomputer-readable medium of program instructions) of the imageprocessing system such as, but not necessarily limited to,microprocessors, digital signal processors (DSPs), field-programmablegate arrays (FPGAs), main memories, secondary/auxiliary memories,input/output devices, operating system software, application software,etc. Any such functions, either entirely or in part, can be furtherimplemented on such a computer-readable medium/media that can be read bythe system to achieve the desired objectives presented herein. Note thatwhile the process functions herein are assigned to discrete processorblocks by way of illustration, it is expressly contemplated thatfunctions of various blocks can be consolidated, expanded to furtherprocessor blocks or reordered between blocks as appropriate to carry outthe overall process described herein.

The image processing system 100 includes a multi-modal, magneticresonance (MR) image source 110 for acquiring and/or storing multi-modalMR image data. In certain embodiments, the image source 110 includes orcomprises an MR scanner apparatus (not shown) that generates imagesusing different magnetic resonance scanning parameters, which are alsoreferred to as sequences, protocols, or scans. An advantage offered byMM is the ability to capture tissue information using differentprotocols within the same acquisition. In other expressly contemplatedembodiments, the image source includes, or comprises, previouslyacquired images saved in an electronic memory or computer-readablestorage medium, such as a computer memory, random access memory (RAM),solid state memory (e.g. compact flash), electro-optical disk storage,magnetic tape or magnetic disk storage, etc. Illustratively, the imagesource can be a Picture Archiving and Communication System (PACS) forstoring, retrieving, and distributing medical image data betweencomponents of the image processing system. Alternatively, any directlyattached or networked storage device with appropriate data organizationcan be employed to store, and allow retrieval of, the image data. Forexample, the storage device can comprise a removable disk or solid-statestorage, a network-attached-storage appliance, a storage area network(SAN) and/or a remote data store accessed by a secure private network,such as a hospital wide area network or a public network. Appropriatelayers of encryption can be applied to the transmitted as well as thestored data as required to satisfy various governmental andinstitutional security requirements. Such encryption techniques shouldbe clear to those of ordinary skill.

In one embodiment, the image source provides a contrast-enhanced medicalimage dataset 112, a T₂-weighted medical image 114, and adiffusion-weighted medical image 116. The source can provide additionalor alternative multi-modal MR medical image data. Other explicitlycontemplated embodiments include, without limitation, T₁-weightednon-fat-saturated medical images and/or MR spectroscopy medical images.The illustrative medical images provided by the source form a “study” ora “case” of organs or tissues under study. Individually, each medicalimage can provide some visual distinction between different tissue types(e.g., malignant or benign). However, some tissue types can only bediscerned using one particular source of image. It has been observedthat the signature of some tissue types is also more apparent whenexamining the image information available through a study. The exemplarymedical images 112, 114, and 116 will now be briefly introduced by wayof illustration.

Dynamic Contrast Enhanced Magnetic Resonance Imaging (DCE-MRI)

In dynamic, contrast-enhanced magnetic resonance imaging, or DCE-MRI,the increased permeability of tumor vasculature gives rise to increasedleakage of tracers such as a contrast medium, which can be administeredto a patient by intravenous injection, or another suitable infusiontechnique. The contrast medium can be any media/agent useful indistinguishing tissue types such as, but not limited to, agadolinium-based contrast agent. In DCE-MR imaging, a stream ofcontrast-enhanced medical image datasets 112 is acquired atpredetermined time intervals (typically before, during, and after thecontrast agent infusion) defined by “dynamic phases.” The streamtherefore enables characterization of visual enhancement patterns in thetissue. By comparing images taken before and after contrast materialinjection, a contrast-enhanced study can help to determine if ananatomical region under study contains abnormalities, whether anabnormality appears benign (non-cancerous) or malignant (cancerous), andthe size and location of any abnormality that appears malignant. It isnoted that contrast-enhanced medical image datasets can be acquiredusing a computed tomography (CT) scanner, in which case the source 110(FIG. 1) can include or comprise a CT scanner apparatus toillustratively acquire this portion of the multi-modal MR image data.

T₂-Weighted MRI

Another MRI sequence of interest is a T₂-weighted (T₂-w) MRI scan. Byway of useful background information, T₂-w MRI scans use a spin echo(SE) sequence with a long echo time (TE) and repetition time (TR) tosuppress imaging of body fat. As a result, different tissue types oftenexhibit different signal intensity values on T₂-w medical images 114.For example, cysts typically appear visually much brighter than otherbreast tissue. Another advantage of the T₂-w MM scan is that the tissuecan be imaged at a higher spatial image resolution than other sequences.

Diffusion-Weighted MRI

By way of useful background information, diffusion-weighted MR pulsesequences measure the apparent diffusion coefficient (ADC) of water intissue. Each voxel of a diffusion-weighted image has a signal intensitythat reflects a single best measurement of the rate of water diffusionat that location. An ADC value associated with each voxel isillustratively computed from a plurality of intensity signals generatedby repeating the diffusion-weighted pulse sequence multiple times.Illustratively, ADC values can be computed and/or provided directly bythe source 110 along with the diffusion-weighted medical image 116.

In an embodiment, the image processing system 100 includes aregistration process/processor 130 that registers multi-modal MR imagesboth intra-(“within”) and inter-(“across”) modality. Suchtransformations can be required due to deformation of the organ ortissue of interest during image acquisition and/or across imagingsequences. Breathing and patient movements are two sources of potentialdeformation. In certain embodiments, the registration process/processorincludes a non-rigid registration process/processor 132 for performingintra-modality registration and an atlas or template registrationprocess/processor 134 for performing inter-modality registration. Inaccordance with further embodiments, the atlas registrationprocess/processor reads organ/tissue atlas coordinate data (also termed,simple “atlas data”) 135 from memory and uses the atlas data tosuccessfully register the images. The atlas data contains a DCE-MRatlas, a T₂-weighted atlas, and a diffusion-weighted atlas. Each atlasrepresents a-priori knowledge about the shape of the anatomical regionin the corresponding sequence. In other embodiments, in which the source110 provides other medical images, different atlases are providedinstead of, or in addition to, those described herein. Illustratively,the stored atlases are pre-registered to one another via an offlineregistration process, which provides advantages during theinter-modality registration runtime procedure. By the transitiveproperty of equality, once each individual image is registered withrespect to its stored atlas, all multi-modal images acquired from thesource are then registered inter-modality. The output from theregistration processor is illustratively shown as registered multi-modalMR image data 139.

In other contemplated embodiments, the registration process/processor130 exclusively includes the non-rigid registration process/processor132 that individually registers each medical image 112, 114, and 116 toa single baseline/reference image. Illustratively, thebaseline/reference image used by the process/processor can be apre-contrast volume (i.e., before contrast injection, also referred toas t₀) within the contrast-enhanced, medical image time series 112. Allother image volumes, including post-contrast volumes of thecontrast-enhanced time series, are then registered to thebaseline/reference image to form the registered image data 139.

In an embodiment, the image processing system 100 includes asegmentation process/processor 140 that automatically segmentsorgan/tissue data of interest 144 from background. Examples ofbackground data can include air, noise, or other tissue that is outsidethe field of view, whereas examples of organ/tissue data of interest caninclude a patient's breast or breasts, lung or lungs, liver, prostate,etc. In accordance with certain embodiments the organ/tissue data ofinterest can be further segmented into distinct regions or zones, whichhas the advantage of enabling a region based or “local” implementationof multi-modal parametric fusion to be further described herein below.For example, in embodiments in which the image source 110 providesmulti-modal images of the prostate and surrounding regions, FIG. 2illustrates exemplary runtime results when the segmentationprocess/processor 140 is configured to segment a central/transition zone(depicted by reference numeral 210) and a peripheral zone (depicted byreference numeral 220) from prostatic images. In other explicitlycontemplated examples and embodiments, the segmentationprocess/processor can segment a prostate peripheral zone, a prostatecentral gland/zone, a prostate transition zone, and prostate seminalvesicles from the image. In one embodiment, the segmentationprocess/processor 140 automatically segments prostatic images into suchillustrative zones using pre-labeled segmentations provided in theorgan/tissue atlas data 135. An example of an atlas-basedregistration/segmentation that can be performed by the registration andsegmentation processes/processors is described by Gubern-Merida et al.in “Atlas Based Segmentation of the prostate in MR images,“International Conference on Medical Image Computing and ComputerAssisted Intervention (MICCAI): Segmentation Challenge Workshop, London,2009,” available through the World Wide Web of the well-known Internetat the URL address,wiki.namic.org/wiki/images/d/d3/Gubern-Merida_Paper.pdf. It is expresslycontemplated that other image processing segmentation techniques can beimplemented in the segmentation process/processor 140.

Note, as used generally herein the terms “automated,” “automatic” and“automatically” should be taken to mean operations that aresubstantially or completely free of human effort in their performance.Such processes are potentially initiated or have parameters set by humaneffort, or by another automated process or apparatus, but thereafterproceed with minimal or no human effort. It is contemplated that, atcertain time points, the automatic process can request human guidance,but the primary work performed in computing and handling data isperformed by the process using, for example hardware and/or software.For example, human effort directs the system to begin fusing image dataand at subsequent times, the system, after performing some or all of thefusion, requests the human operator to enter a location to store ordisplay the results.

In an embodiment, the image processing system 100 (FIG. 1) includes anintensity inhomogeneity correction process/processor 150 that correctsmedical images for intensity inhomogeneity artifacts occurring as aresult of non-homogeneous magnetic fields during image acquisition. Byway of background, subtle variations in the strength of the appliedmagnetic field with respect to location can typically occur. The appliedmagnetic field can also vary with respect to time. The effect of theinhomogeneous magnetic field can result in uneven enhancement throughoutportions of the image that should otherwise appear uniform. Thisunevenness can adversely affect further post-processing of the images.For example, in the contrast-enhanced medical images 112, the variationsin the magnetic field can falsely simulate the appearance of signalenhancement over time. One algorithm suitable for implementation in theintensity inhomogeneity correction processor is described by way ofuseful background in Salvado, et al., in “Method to correct intensityinhomogeneity in MR images for atherosclerosis characterization,” IEEETrans Med Imaging, 2006 May; 25(5):539-52. It is expressly contemplatedthat other algorithms may also be implemented in the intensityinhomogeneity correction process/processor. The output from theprocessor is illustratively shown as homogeneous multi-modal MR imagedata 155.

The image processing system 100 further includes an image fusionprocess/processor 160 that generates a fused response image or map 166from the multi-modal MR image data. For purposes of this description,the fused response image can also be defined generally as a “probabilitymap” or “color map.” Briefly, “image fusion” as used herein can bedefined as a process of combining raw image data, post-processed imagedata, or combinations thereof from two or more images acquired usingdifferent medical imaging (e.g., magnetic resonance) scanner parameters(i.e., sequences, protocols, scans) into a single image. The “responseimage,” “probability map,” and “color map” are examples of singularimages that visually illustrate data resulting from an image fusionprocess. The single image thereby provides an important diagnosticoutput tool for a clinician, as it synthesizes a large amount ofspatial, temporal, and functional tissue data in an objective manner.This tool enables a clinician to quickly and easily visualize tissuepixels, voxels, or regions of potential interest without necessarilyhaving to review an entire case of MR images, which can be atime-consuming, subjective, and error-prone process. Exemplary processesby which the image fusion processor can generate the fused responseimage are further described herein below.

In an embodiment, the image processing system 100 is connected to aninput device 170, and enables a user-defined implementation ofmulti-modal parametric fusion. The input device can be a keyboard, amouse unit, a touch screen/touchpad, a voice-activated input device, orother suitable device that can be conventional or customized in a mannerclear to those of skill in the art. The system receives a set ofuser-defined parameters 172 via the input device. The parameters 172 areillustratively stored in memory (not shown in FIG. 1) in the processingsystem, and transmitted to the image fusion process/processor 160 atruntime to be operated upon in the generation of a desired outputresponse image. It is expressly contemplated that the user-definedparameters can also be received in a flat file, database or othersuitable data structure via a network connection (not shown in FIG. 1)from other systems, allowing different clinicians to remotely share andimplement different fusion parameters.

It can be desirable to enable human users to input parameters to be usedin the multi-modal MR image fusion procedure so as to achieve a varietyof goals. There is no universally accepted way in the medical communityfor clinicians to combine and evaluate MRI datasets acquired withdifferent sequences. As discussed in the background, differentclinicians possibly prefer to evaluate images in different manners.Experimentation with different types, weights and combinations of inputparameters can enable clinicians to discover optimal parameters forcomputing novel response images. For example, one set of input weightscan be used for creating response images optimal for distinguishingmalignancy tissue from benign tissue. Another set of input weights canbe used for creating response images optimal for identifying aparticular type of malignant tissue, such as ductal carcinoma in situ(“DCIS”) breast cancers. Another set of input weights can be used forcreating response images optimal for identifying cancer in a particularregion of an organ. Furthermore, experimentation with and modificationsto input parameters can be desirable, or required, as changes to MMprotocol and sequences are made to the clinician's MR image acquisitionprotocols.

Another component of the image processing system 100 is an output device180. The output device can comprise a printer, a computer monitor, aseries of computer monitors, and/or other suitable signal output devicesof conventional or novel design that generate one or more viewableimages. The output device could also be a compact disc, disk driveand/or another suitable storage medium/device that allows image data tobe stored, accessed and output. The output device enables signals 185 tobe output in many forms suitable for visual inspection of the tissueincluding, without limitation, raw images 112, 114, and/or 116 acquiredby the source 110 and fused response images 166 produced by the imagefusion process/processor 160.

It is expressly contemplated that components of the image processingsystem 100 can connect to and communicate with each other via one ormore of any type or combination of types of communication interfaces,including but not limited to physical interfaces, network interfaces,software interfaces, and the like. The communication can be implementedusing a physical connection, and/or wireless, optical, or any othermedium. Alternatively, image datasets and/or fused response images canbe transmitted indirectly by use of transportable storage devices (notshown in FIG. 1) such as but not limited to portable compact discs(CDs), digital video discs (DVDs), or solid state “flash” drives, inwhich case readers for said transportable storage devices can functionas communication interfaces of the system.

Runtime Operation of the System

In operation, and with further reference to FIG. 1 which will be used todescribe the steps in a runtime procedure, the contrast-enhanced medicalimage 112, the T₂-weighted medical image 114, and the diffusion-weightedmedical image 116 are acquired from the source 110, and loaded into asystem image memory for processing. In a next step, the non-rigidregistration process/processor 132 registers the volumes of thecontrast-enhanced, medical image 112. The atlas registration processor134 then reads the organ/tissue atlas data 135 from memory andindividually registers the intra-registered contrast-enhanced medicalimage, T₂-weighted medical image, and diffusion-weighted medical imageto their respective stored, pre-registered atlases. After theregistration step, the tissues and/or organs in the multi-modal MRimages are considered aligned in a single coordinate system. In a nextstep, the segmentation processor/processor 140 applies prelabelled imagecoordinates of the organ/tissue atlas data 135 to each registered imageas a mechanism to automatically detect and segment the organ/tissue ofinterest 144. Optionally, the organ data is further segmented intodistinct regions or zones. This can allow focus on a particular area orareas of the tissue that is of specific interest to the clinician. In afurther optional procedure, the intensity inhomogeneity correctionprocess/processor 150 corrects non-homogenous image intensities of thesegmented organ/tissue of interest data or subsets thereof. This can beperformed using conventional image smoothing and filtering techniques,among other image processing applications. In alternate embodiments, theorder of the steps of registration, segmentation and intensityinhomogeneity image pre-processing can be varied. For example, intensityinhomogeneity correction can be performed prior to registration orsegmentation. In the next step, the pre-processed image data is thentransmitted to the image fusion process/processor 160 for computation ofthe fused response image 166, examples of which are referenced in detailherein below.

Image Fusion Process/Processor

With reference now to FIG. 3, a block diagram of an illustrativeembodiment of the image fusion process/processor 160 is shown. Withreference also to FIG. 1, the inputs are pre-processed representations(e.g., registered, segmented, inhomogeneous intensity corrected) of theimages 112, 114, and 116 to be fused. In embodiments of a user-definedimplementation of the fusion process, user-defined parameters 172 arealso acquired as inputs. The output is the fused response image and/orthe probability map 166 of computed response values.

The image fusion process/processor 160 includes a multi-modal parametersource 310 for acquiring and/or storing multi-modal parameter values orfeatures to be used in fusion computations. In certain embodiments, rawvalues of some or all of the input images are provided as parametervalues. The input images can be smoothed before fusion to remove noiseand/or provide more continuous parameter values. Illustrative examplesof parameter values include a T₂-weighted intensity map 312 and an ADCmap 314. Examples are visually illustrated in FIGS. 4 and 5. “Map” or“parametric map” as used herein refers to a representation of parametricfeature values associated with the pixels or voxels of an image.

FIG. 4 provides an exemplary T₂-weighted intensity map 400 that wasderived by smoothing, scaling, and inverting a T₂-weighted image of aprostate 410 and surrounding regions in accordance with illustrativeoperational steps to be further described herein below. Using theseinverted maps, the image fusion process/processor 160 can provide agreater emphasis in tissues that exhibit high signal intensity values.In this example, within the general region of the prostate, althoughdifferences in signal intensity values can be observed, there appears tobe no conclusive indication of a region of interest.

FIG. 5 provides an exemplary ADC map 500 that was derived by smoothing,scaling, and inverting a diffusion-weighted image of the same prostatedepicted in FIG. 4 in accordance with illustrative operational steps tobe further described herein below. Using these inverted maps, the imagefusion process/processor 160 can provide a greater emphasis in tissuesthat exhibit high signal intensity values. In this example, note acluster of pixels depicted by the reference numeral 510 that appear tobe a potential tissue region of interest, as the clusters appear to bequite light relative to surrounding tissues in the prostatecentral/transition zone. Referring back to the T₂-weighted intensity map400, only a slight difference between intensities can be observed inthis region with respect to other prostate tissues.

In certain embodiments of the image fusion process/processor 160 asshown in FIG. 3, the parameter source 310 optionally includes orcomprises a parameter or feature extraction process/processor 315 thatextracts parametric values from some or all of the multi-modal inputimages. Illustratively, physiological or kinetic tissue parameters canbe extracted by applying a known pharmacokinetic model (e.g., Toftsmodel) to the pixel signal intensity values of the pre-processed,contrast-enhanced medical image dataset. Exemplary kinetic tissueparameters that can be extracted from such imagery include a transferconstant or permeability surface (k_(trans)) map 318, an extracellularvolume (v_(e)) map 319, and/or an efflux rate constant (k_(ep)) map 320,examples of which are visually illustrated as derived images or maps inFIGS. 6, 7, and 8, respectively. In one embodiment, the parameterextraction process/processor 315 automatically computes such kinetictissue parameters as described by Tofts et al., in “Estimating kineticparameters from dynamic contrast-enhanced T(1)-weighted Mill of adiffusable tracer: standardized quantities and symbols,” J Magn ResonImaging, 10 (3). pp. 223-232.

FIG. 6 provides an exemplary smoothed permeability map 600 that has beenderived by computing permeability parameters from a contrast-enhanceddataset of the same prostate depicted in FIGS. 4-5 in accordance with anembodiment. Using these maps, the image fusion process/processor 160 canprovide a greater emphasis in tissues that exhibit high signal intensityvalues. In this example, the reader should note two clusters of pixelsdepicted by the reference numerals 610 and 620 that appear to bepotential regions of interest, as the clusters exhibit strongintensities. While cluster 620 has failed to exhibit interestingsignatures in both the T₂-weighted intensity and ADC maps, cluster 610appears to be the same potential tissue region of interest as thecluster 510 in ADC map 500.

FIG. 7 provides an exemplary smoothed extracellular volume map 700 thathas been derived by computing extracellular volume parameters from acontrast-enhanced dataset of the same prostate depicted in FIGS. 4-6.Using these maps, the image fusion process/processor 160 can provide agreater emphasis in tissues that exhibit high signal intensity values.In this example, within the general region of the prostate, thereappears to be no conclusive indication of a region of interest, althoughtissues on the left side of the prostate central/transition zone doappear to be lighter relative to tissues on the right side.

FIG. 8 provides an exemplary smoothed efflux rate constant map 800 thathas been derived by computing efflux rate parameters from acontrast-enhanced dataset of the same prostate depicted in FIGS. 4-7.Using these maps, the image fusion process/processor 160 can provide agreater emphasis in tissues that exhibit high signal intensity values.In this example, the reader should note a cluster of pixels depicted bythe reference numeral 810 that appear to be lightest relative tosurrounding tissues. Cluster 810 appears to be the same potential tissueregion of interest exhibiting a strong response in the ADC map 500 andthe permeability map 600. Clearly, the clinician can benefit from afusion image that directs attention to this region based on the pluralresponses exhibited in the parametric maps.

Other explicitly contemplated embodiments of parameters that can becomputed by the parametric feature extraction processor 315 include,without limitation, time-to-peak parameters, contrast uptake or wash-inrate parameters, contrast washout rate parameters, peak enhancementparameters, slope parameters, and/or initial area under the contrastagent concentration curve (iAUC).

Other exemplary parametric values or features for extraction includemorphological shape parameters in the form of a morphology map 322.While kinetic parameters characterize the motion of contrast agent intissues, morphological parameters characterize the form or structure ofthe tissues. In one embodiment, morphologic shape parameters can beextracted by applying a Gaussian-based blob detection algorithm topre-contrast, spatial pixel signal intensity values of thepre-processed, T₂-weighted medical image. Other explicitly contemplatedembodiments of morphological maps or parameters include, withoutlimitation, texture parameters for characterizing sharp or fuzzy edgesof tissues.

In an embodiment, the image fusion process/processor 160 (FIG. 1)includes a scale normalizing process/processor 330 (FIG. 3) thatnormalizes input fusion parameters to a single scale. According to oneembodiment, a pre-defined scale of 0 to 1 can be used by the processor,which emulates a probability range between 0 and 100%. That is, forevery input parameter value, the scale normalizing processor provides anew output value in the range of 0 to 1. The conversion can be performedon a per-voxel level for all input fusion parameter values. Themechanism by which normalization occurs is highly variable. In general,normalization of a series if values entails analysis by theprocessor/process of the parameter's minima and maxima and thereafterapplying a scalar that places the overall range of values within anormalized range that is acceptable to all images within the set beingfused. In an embodiment, the scale normalizing process/processor ignoresatypical observation values (e.g., above the 98th percentile, below the2nd percentile) to ensure that outliers do not negatively affect thenormalized values with respect to scale. The output from this process isillustratively shown as scale normalized parameter values 335.

In an embodiment, the image fusion process/processor 160 (FIG. 1)includes a response or probability value process/processor 340 (FIG. 3)that combines the scale-normalized parameter values to form finalresponse values 345 that define the fused response image 166. Inembodiments of a user-defined implementation of the fusion process, theuser-defined parameters 172 are used as inputs in the computation of thefinal response value, which advantageously provides a more customizedand intuitive fusion image to the reviewer. In embodiments of theregion-based or “local” implementation of the fusion process, thescale-normalized parameter values are combined distinctively inaccordance with the segmented organ zone in which they appear. Forexample, parameter values in the prostate peripheral zone can beweighted more heavily than the parameter values in the prostate centralgland. In this embodiment, the response values are biased with a highersensitivity towards regions of the tissues/organs where malignancies aremore likely to appear. This biasing can be accomplished by applying amap of regions that includes predetermined information on the locationsof such regions in a typical image. That is, where a region is likely toappear in a given portion of the image, that portion is flagged and theprocessed image is biased toward the corresponding region.Alternatively, pattern recognition or image processing applications candetermine the location of the region or regions in which bias is tooccur. In further embodiments of both user-defined and region-basedimplementations, clinicians are provided the flexibility of imposingregion-specific biases to sequence- or parameter-specific imageinformation. This can be accomplished by providing a region selectionfunction in the user interface. The region selection function caninclude a menu or display of available regions, and selection buttonsand/or slides (or other interface elements) that allow the selection ofbias as well as the degree of bias.

The image fusion processor 160 illustratively utilizes either a linearor a non-linear function to compute the final response values 345. Thefusion computation can be performed on a per-voxel basis and repeatedfor every voxel within the pre-processed image data, thereby creating acollection of response values suitable for implementation in theprobability map 166.

Reference is now made to an illustrative, multi-modal MR image fusionruntime procedure 900 as shown in FIG. 9. The various process steps inthe procedure 900 have been grouped by dash-line boxes into a pluralityof corresponding process/processors as described above with reference toFIG. 3, and associated reference numbers have been provided for eachdashed-line box. It should be noted that the depicted steps can bearranged alternatively so that they are performed within a differentprocess/processor, or performed in a discrete process/processor otherthan those described in FIG. 3.

At step 910, a plurality of parametric values to be fused is acquired.In this embodiment, the values include, on a per-voxel basis,T₂-weighted signal intensities, apparent diffusion coefficients,permeability parameters, and extracellular volume parameters.Illustratively, some values are acquired via the multi-modal parametersource 310, while some other values are computed by the parameterextraction processor 315.

Each discrete parameter value can potentially lie within a differentrange of values with respect to the other image parameter values. By wayof example and without limitation, each voxel can have T₂-weightedsignal intensity values ranging between 0 and 1,200, ADC values rangingbetween 0 and 5, extracellular volume values ranging between 0 and 1,and permeability values ranging between 0 and 2. At step 915 of theprocedure 900, the values are converted to lie between an illustrativenormalized scale of 0 to 1. Such steps can be performed by the scalenormalizing process/processor 330, in an illustrative embodiment. Forpurposes of this description, each normalized set of values can bereferred to at runtime as an “intermediate probability map” andillustrative examples of intermediate probability maps are depicted inFIGS. 4-8.

It is noted that, in certain parametric input data such as T₂-weightedsignal intensities, as the intensity signal value of a voxel movescloser to 0, the voxel might be of a higher interest because low signalintensity values more frequently indicate tissue of interest (e.g.,malignancies). At step 920, an inverse function is applied thattransforms lower signal values into higher normalized probability valuesand vice versa. Such steps can also be illustratively performed by thescale normalizing process/processor 330. This step can be alternatelyperformed prior to the scale normalization of the parameter values, inwhich case the inverted values are then normalized.

At step 925, a discrete map of response values is computed from theplurality of intermediate probability maps. Such steps are performed bythe response value process/processor 340, in an illustrative embodiment.In certain embodiments, it is desirable to employ a linear function inthe computation of the response values because the processing used bythe system to derive the final probability map will become relativelyintuitive to a clinician. That is, the clinician can readily comprehendthe impact of each individual parameter value on the final responsevalue. Furthermore, in embodiments of a user-defined implementation ofthe fusion process, the clinician can readily comprehend the impact thatmodifications to the user-defined parameters 172 will have on theresponse values.

An exemplary linear function for computing response values is shown byEquation 1:

$\begin{matrix}\frac{\sum\limits_{i}\;{w_{i}{P\left( m_{i} \right)}}}{\sum\; w_{i}} & \left( {{Eq}.\mspace{14mu} 1} \right)\end{matrix}$

where P(m_(i)) corresponds to a specific intermediate probability map ofan image, and i and W_(i) each correspond to a weight associated withP(m_(i)).

In other embodiments, a non-linear function can be used to computeresponse values, in which the weighted probability maps are multipliedto compute a response value. An exemplary non-linear function forcomputing response values is shown by Equation 2:

$\begin{matrix}{\prod\limits_{i}\;{w_{i}{P\left( m_{i} \right)}}} & \left( {{Eq}.\mspace{14mu} 2} \right)\end{matrix}$

At step 930 of the procedure 900, once response values are computed foreach voxel, this data can be illustratively presented to the clinicianin a raw image (e.g., by enhancing the intensities of a grayscale imagein accordance with response values that indicate regions/voxels ofinterest), through a color map, or other conventional or novel visualpresentation known to one of skill. In certain embodiments, a hot colormap is formed and presented in which response values moving toward therange bound of 1 are colored red, values moving toward the opposingrange bound of 0 are colored blue, and continuous values in between arecolored with different shades that can conform with the spectrum oranother appropriate color scale. Thus, it is expressly contemplated thatthe illustrative range bounds and associated colors are merelyexemplary. Other choices could be implemented. It is furthercontemplated that voxels having response values below a cutoff thresholdare not colored. The specific bounds, color intensities, color hues,and/or thresholds can also be specified as input parameters by theclinician via a system interface.

The results of a fusion process according to the system and methoddescribed herein are shown, by way of example in FIG. 10. The depictedfused response image provides a hot color map 1000 overlaid on anillustrative T₂-weighted prostate MR image 1010, although the hot colormap could be presented as an overlay on a different image. The large(red) region 1020 signifies an area of concern in the subject prostateillustratively depicted in the parametric maps 400, 500, 600, 700, and800. Thus, by providing a fusion of multiple images, each representingdiffering parameter sets, the clinician's attention is directed to areasof concern by consulting a single fused dataset. Moreover, thepresentation of features in such areas is customized to reveal thesefeatures in a manner that most effectively assists the clinician inperforming his or her analysis.

CONCLUSION

It should be clear that the system and method of the illustrativeembodiments provide a fused magnetic resonance image that helpsclinicians to more quickly and more accurately evaluate a significantamount of multi-modal MR image data information.

The foregoing has been a detailed description of illustrativeembodiments of the invention. Various modifications and additions can bemade without departing from the spirit and scope of this invention. Eachof the various embodiments described above may be combined with otherdescribed embodiments in order to provide multiple features.Furthermore, while the foregoing describes a number of separateembodiments of the system and method of the present invention, what hasbeen described herein is merely illustrative of the application of theprinciples of the present invention. For example, some or all of theprocesses described herein can be implemented in hardware, software,including a computer-readable medium of program instructions, or acombination of hardware and software. Moreover, while images of ananatomical prostate have been presented to illustrate various aspects ofthe illustrative embodiments, such images should not be construed aslimiting the utility of the present invention to any one particularorgan or tissue. Accordingly, this description is meant to be taken onlyby way of example, and not to otherwise limit the scope of thisinvention.

What is claimed is:
 1. A system for fusing and synthesizing a pluralityof medical images defined by a plurality of imaging parameters, thesystem comprising: an image fusion processor that fuses a plurality ofmagnetic resonance imaging (MRI) datasets; wherein the image fusionprocessor generates a fused response image that includes a combinationof image features from a first image dataset and image features from asecond image dataset based upon user defined parameters input to theimage fusion processor through a user interface prior to operation ofthe image fusion processor to generate the fused response image; and asegmentation processor, in communication with the image fusionprocessor, that generates corresponding segments of each of the firstimage dataset and the second image data set based upon predeterminedsegmenting data so that each of the corresponding segments are eachdiscretely fused by the image fusion processor.
 2. The system as setforth in claim 1, wherein the plurality of MRI datasets includes: afirst image dataset defined by at least a first parameter and a secondimage dataset defined by at least a second parameter.
 3. The system asset forth in claim 2, wherein the first parameter comprises morphologyvalues and the second parameter comprises a parameter based upon atleast one of a T2-weighted medical image, a dynamic contrast enhancedMRI (DCE-MRI) medical image, and an image based upon apparent diffusioncoefficient (ADC).
 4. The system as set forth in claim 1, wherein thepredetermined segmenting data is based upon organ/tissue atlascoordinate data.
 5. The system as set forth in claim 4, furthercomprising a registration processor that generates multi-modalregistered image data from each of the first image dataset and thesecond image dataset.
 6. A non-transitory computer-readable mediumstoring instructions readable and executable by a computer to perform amethod for fusing and synthesizing a plurality of medical images definedby a plurality of imaging parameters, the method comprising: fusing aplurality of magnetic resonance imaging (MRI) datasets including atleast a first image dataset and a second image dataset wherein thefusing includes generating a fused response image that includes acombination of image features from the first image dataset and imagefeatures from the second image dataset based upon user definedparameters input through a user interface prior to the generating of thefused response image; and generating corresponding segments of each ofthe first image dataset and the second image data-set based uponpredetermined segmenting data so that each of the corresponding segmentsare each discretely fused.
 7. A method for fusing and synthesizing aplurality of medical images defined by a plurality of imagingparameters, the method comprising: fusing a plurality of magneticresonance imaging (MRI) datasets including at least a first imagedataset and a second image dataset wherein the fusing includesgenerating a fused response image that includes a combination of imagefeatures from the first image dataset and image features from the secondimage dataset based upon user defined parameters input through a userinterface prior to the generating of the fused response image; andgenerating corresponding segments of each of the first image dataset andthe second image dataset based upon predetermined segmenting data sothat each of the corresponding segments are each discretely fused.